Skip to content

Develop - #1345

Merged
amirsadraabdollahi merged 19 commits into
masterfrom
develop
Jul 15, 2026
Merged

Develop#1345
amirsadraabdollahi merged 19 commits into
masterfrom
develop

Conversation

@amirsadraabdollahi

Copy link
Copy Markdown
Contributor

No description provided.

# Modernize backend: Django 4.2 · Celery 5 · Python 3.11 · PostgreSQL 16 · TF 2.15 / CUDA 12

## Summary
Brings Rodan's backend off its end-of-life stack and onto current, H100-capable infrastructure. Upgrades the web/framework layer, the database, and the GPU job runtime, adapts the container/k8s deployment accordingly, and fixes a set of regressions surfaced by the upgrades. 18 commits, ~129 files.

## Motivation
The old stack (Django 2.0, Celery 4.4, Python 3.7, PostgreSQL 9.6, and CUDA/TF builds targeting pre-Hopper GPUs) no longer installs cleanly, receives no security fixes, and can't run on the current H100 (sm_90) hardware. This branch is the coordinated upgrade of all of those at once.

## What's changed

**Framework (Phase 1)**
- Django 2.0 → **4.2.16**, DRF → **3.15.2**, djoser → 2.2, Celery 4.4 → **5.3.6**, Python 3.7 → **3.11**, numpy → 1.26.4.
- Realtime layer migrated from `ws4redis` → **Django Channels 4** (ASGI `ProtocolTypeRouter` + a broadcast consumer relaying the redis pub/sub channel).
- `gamera4` rebuilt for py3.11.

**Database**
- PostgreSQL 9.6 → **16** (required by Django 4.2). Includes a dump/restore cutover runbook (`k8s/POSTGRES_UPGRADE.md`); `postgres/Dockerfile` now `FROM postgres:16`.

**GPU jobs (Phase 3)**
- GPU job image rebuilt on **TensorFlow 2.15 / CUDA 12 / Python 3.11** for H100.
- `text_alignment` ported to py3.11 / TF 2.15 (upstream calamari-ocr 2.3.1).

**Containerization / k8s**
- Image chain rebuilt (`rodan-python3-celery` → `rodan-main` → `nginx`; plus client, gpu-celery, postgres, iipsrv) and all k8s manifests updated; GPU pod gets MIG visibility and `compute,utility` capabilities.

**CI/CD**
- `build-and-deploy` and `build-ci-jobs` workflows; CI test fixes.

**Bug fixes surfaced by the upgrade**
- **List filtering (`filter`→`filterset`)** — django-filter 24.3 ignores the legacy `filter_fields`/`filter_class`; renamed to `filterset_fields`/`filterset_class` across all list views (restores `?project=` and other filters), fixed 3 filter-method lambda signatures, and **restored object-level list scoping** via a guardian-based `ObjectPermissionsFilter` in `DEFAULT_FILTER_BACKENDS` (a list with no `?project=` no longer leaks every project's rows).
- **Interactive editor open** — per-key template cache fix so a second editor open per worker doesn't 500.
- **Job import guard**, **ASGI startup**, `is`→`==` identity check, gpu-celery CUDA env.

## Breaking changes / migration notes
- **Requires the PostgreSQL 9.6→16 dump/restore** before deploy (see `k8s/POSTGRES_UPGRADE.md`).
- **All images must be rebuilt** — base-image chain, Python, and CUDA/TF versions all moved.
- Deploy is triggered by a `v*` tag or a manual `build-and-deploy` dispatch (merging to `develop` only builds `nightly`/`sha-` images).

## Testing & validation
- CI test suite green on the branch.
- All automated job types validated against real DB rows + PVC files via a replay harness; both interactive editors (Neon, Pixel.js) verified end-to-end in a real browser on the modernized stack.
- Live distributed end-to-end workflow run passed on the k3s cluster.
@github-actions

Copy link
Copy Markdown

SHA: 86be539

details
- Fix: change filter to filterset
- Fix: editor-open
- Fix: guard job_list bookkeeping so bare job-module imports don't raise spurious AttributeError
- Feat: port text_alignment GPU job to py3.11 / TF2.15 (upstream calamari-ocr 2.3.1)
- Feat: Add test for SAE
- Fix: change 'is' to '=='
- Fix: Add cuda env to gpu-celery manifest
- Fix: ci tests
- Feat: add build and push workflow for ci-jobs
- Refactor: prepare for merge into develop
- Fix: make the MIG visible to gpu pod
- Fix: add compute and utility to capabilities in GPU
- Fix asgi startup
- Fix: chain image build
- Feat: change k8s images to modernize branch, fix chain images build
- Feat: Phase3 TF 2.15 / CUDA 12 / py3.11 GPU jobs
- Feat: postgres update
- Feat: Phase1 migration the core upgrade of django and python

@amirsadraabdollahi
amirsadraabdollahi merged commit 0c75017 into master Jul 15, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant